Integrated MCDA: A Simulation Case Study

Despite the fact that the origins of the MCDM school lie in extensions of linear programming to problems with multiple objectives, multicriteria analysis, in particular methods for choosing between discrete alternatives, tend to be viewed by Operational Research/Management Science (OR/MS) practitioners as distinct from other methods of OR/MS. In this paper we argue that MCDA should be viewed as an integral part of all problem solving methodologies, not as a set of tools applicable only to certain categories of problem. We will illustrate our argument by reference to a problem faced by the management of a regional UK airport, namely the allocation of check-in desks to airlines. In this case study simulation and multicriteria analysis have been integrated, using the softwares Simul8 and V•I•S•A , to provide a decision support tool for the airport Planning Manager. The DSS will be used in the short term to allocate check-in time slots in a way which achieves effective management of queues and utilization of the desks; in the longer term it will inform the allocation of take-off times. Keywords, Simulation, Multi-attribute value theory, Integrating methods Introduction Amongst the many influences on and contributions to the development of multicriteria analysis, a term which we use to embrace all multicriteria approaches, it is possible to identify three clear threads the MCDM/MOLP school having its origins in extensions of linear programming to problems with multiple objectives, the US dominated field of decision theory, incorporating multiattribute value/utility theory, and the French led school of outranking methods. Despite the fact that, at least, the first two of these represent apparent attempts to broaden the scope of single objective modelling approaches to cope with the added complexity which is characterised in practice by multiple, conflicting objectives, multicriteria analysis, in particular methods for choosing between discrete alternatives (which 1 Address for correspondence Dr Valerie Belton, Management Science, University of Strathclyde, 40 George Street, Glasgow G1 1 QE, UK Tel: 44 141 548 3615, Fax: 44 141 552 6686, Email: [email protected] we will refer to as MCDA methods), is generally viewed by Operational Research / Management Science (OR/MS) practitioners as distinct from other methods of OR/MS. In this paper we argue that multicriteria analysis should be viewed as an integral part of all problem solving methodologies, not as a set of tools applicable only to certain categories of problem. In the following section we expand on this argument and comment briefly on the extent to which integration is reflected in the literature. In section 3 we describe a case study in which simulation and multicriteria analysis are used in an integrated way to examine the problem of allocating check-in desks to airlines at a UK regional airport. In section 4 we reflect on the lessons from this experience and suggest avenues for further investigation. Multicriteria Analysis and OR/MS In considering the extent of integration of multicriteria analysis and OR/MS it is helpful to distinguish between MOLP methods for the exploration of a continuous solution space in which alternatives are implicitly defined by a set of constraints (MCDM), and methods for the evaluation of and choice between a finite set of discretely defined alternatives (MCDA). It is the second group of approaches, comprising multi-attribute value theory, AHP, outranking methods, etc., which are of particular interest in this paper. MOLP and OR/MS We have already noted that the MCDM school has its origins in the extensions of linear programming to problems with multiple objectives, thus, in one sense we can view MOLP as the integration of linear programming and multicriteria analysis. There are many reported applications of MOLP, in particular of goal programming. However, rather than being viewed as an integrated part of optimisation, we feel that MCDM has developed as a separate area of study, as something that is distinguished from single objective optimisation rather than a natural extension of it. This is a concern which is echoed in current discussions in the multicriteria community, as highlighted by Korhonen (1997) in the Opinion Makers Section of the newsletter of the European Working Group (EWG) for “Multicriteria Aid for Decisions”. MCDA and OR/MS A cursory examination of the literature and practice of OR/MS suggests that MCDA is viewed as a stand-alone body of theory, or set of tools, in the same way as, say, simulation or forecasting methods. This sense of isolation is reflected in published accounts of both practical applications and theoretical developments. The latter tend to focus on introspective issues and the majority of reported practical interventions are accounts of “stand-alone” applications of MCDA methods. This is supported by a review of the contents of the Journal of MultiCriteria Analysis and the published proceedings of these Conferences (International Special Interest Group on MCDM). Only a small number of studies draw on, or incorporate MCDA within the broader repertoire of OR/MS approaches to problem structuring and analysis. Examples are: the paper by Spengler and Penkuhn (1996) describing a DSS which combines a flowsheet-based simulation with multicriteria analysis: work by Macharis (1997) which incorporates a multicriteria choice rule within a system dynamics analysis of transport policy: and the study by Gravel et al (1991) using multicriteria analysis to evaluate production plans developed using simulation. It is interesting to note that the isolation is bi-directional; MCDA analysts do not seek to make use of other OR/MS methodologies but neither do OR/MS practitioners specialising in other methodologies draw extensively on MCDA methods. This issue is addressed by Belton and Pictet (1997) in a later Opinion Makers Section of the EWG newsletter, commenting on the views expressed by Korhonen (1997) and others. Clearly there is an important role for MCDA in “stand-alone” applications and we do not wish to detract in any way from this. Many decisions present themselves as a need to choose between a number of well specified alternative courses of action tender evaluation, personnel selection and equipment selection to give just a few examples. However, we feel that MCDA can and should be much more widely used by OR/MS practitioners in conjunction with other analytic tools. Ways of combining MCDA and OR/MS There are many ways in which the use of MCDA in conjunction with other OR/MS methods can lead to mutual enhancement. Exploring these possibilities, with the aim of expanding the influence of MCDA and its acceptance as a valuable tool for OR/MS practitioners, has been a particular interest of one of the authors. This work has identified a number of ways in which MCDA can work synergistically with other approaches: Belton, Ackermann and Shepherd (1995) discuss the use of the SODA methodology together with MCDA to give an integrated approach from problem structuring through to evaluation: Belton and Elder (1996) describe a visual interactive DSS for production scheduling in which multicriteria analysis is embedded in the scheduling algorithm: Belton and Vickers (1993) discuss the parallels between MCDA and DEA and put forward the suggestion that an MCDA interpretation of DEA can facilitate understanding. We refer to these three “combinations” as integrating, embedding and exploring parallels. In this paper we explore further possibilities for the integration of MCDA and other approaches, focusing here on simulation. Background to the Case Study Most airports in the world face the problem described here that of allocating check-in desks to flights and airlines. We describe ongoing work to assist the planning manager of a major UK airport (the 5th busiest in the UK). Significance of the problem Like many decisions facing any management this decision is influenced by many factors: it is affected by, and itself influences, other decisions. The average time passengers queue to check-in for each flight along with the worst case of queuing time for each flight are key factors which drive the decision. However, there are other considerations for example: the length of the queue (which is only partially correlated with queuing time): the space allocated to each airline: and the details of queuing times for different types of flight such as business / holiday, and domestic / international / long haul flights. The length of time prior to takeoff that a checkin can be opened is also relevant. The aim of the planning manager is to allocate flights to check-in desks in a way, which achieves effective management of queues and utilisation of the desks. In doing this he aims to meet the airports own performance measures, which are aligned with meeting the objectives of the airlines and passengers. At one level this is a weekly decision about which flights will use which check-in desks; however, it also has a longer-term impact on the allocation of takeoff times to flights. Airlines state their preferences for takeoff times at a bi-annual conference at which all international airlines and airports get together to coordinate plans for timetables which will be compatible with both the airlines’ commercial demands and the airport capacities. Runway capacity is clearly an important factor in determining capacity, but the capacity of the check-in hall is in fact much more significant for our decision maker. For example, the check-in facilities would have some difficulty in coping if more than four 747’s (or aircraft of equivalent size) were scheduled to leave within half an hour, although this presents no problems with respect to runway capacity. The decision is taken weekly by a planner who is directly responsible to a member of the airport’s senior management team. It is this member of the management team who has to deal with subsequent complaints, for example: complaints from airlines unhappy with their allocation: complaints from passengers about the length of queues: complaints from passengers about missing flights because of queuing times: or complaints about the overcrowding of the check-in hall. The airport and its parent company place a very high emphasis on customer service and getting this decision right is seen as a key element in the process of increasing the use of the airport (and thus its profitability). Consequently, the responsible member of the management team monitors the situation very closely. The physical environment Each check-in desk is identical. The sign on each check-in desk is a video screen, which can be easily and automatically changed to show the currently allocated airline and, if appropriate, flight number. The schedule for the coming week is programmed once the allocation decision has been taken. There are 34 check-in desks and around 800 passenger carrying flights leave the airport in a peak week. Around 40 airlines operate from the airport and of these only 2 or 3 have the need for a permanently allocated desk. Others operate only a few flights each day, or in some cases 1 or 2 flights per week. A holiday charter flight to a long-haul destination, with a large number of passengers, needs to open one check-in up to five hours before takeoff time. For the period between 3 to 2 hours prior to departure there may be 4 desks allocated to the flight, the number reducing as the flight time approaches. The consequence of this pattern of demand is that most check-in desks must be reallocated continually. Some airlines are prepared to have several of their flights sharing one or more check-ins, but others prefer to dedicate check-ins to particular flights. When an airline is allocated several check-in desks at one time they have a strong preference for all of these to be adjacent. Some airlines have a strong preference to keep the same check-in desks for the entire week, or at least to have desks in a group which passengers will come to recognize as approximately the area to use for that airline. Some airlines are prepared to pay extra to be allocated more desks than they really deserve in order to reduce queues for their passengers or, in the case of one airline, simply to make it appear that they are a larger airline than is actually the case! Current decision process The decision on how to allocate check-in desks is taken by the same person each week drawing on past experience. Prior to this study the only tool available to the planner was a spreadsheet into which he entered proposed check-in allocations taking into account scheduled take-off times. The spreadsheet is illustrated in Figure 1: each row corresponds to a check-in desk and each column to a 30 minute time interval. The planner enters into each cell in the matrix the airline to be allocated to a given desk at the specified time. It is simply a visual presentation tool; no analysis is provided. At certain times of year there will be few changes from one week to the next, but changing schedules necessitate some change in most weeks. When the planner arrives at an allocation he is satisfied with the spreadsheet is emailed to the airlines and other interested parties. Airlines can quickly see their allocation and are quick to complain if they believe this to be unfair. However, without the support of an analytic technique it is difficult to assess the likely impact of an allocation of check-in desks on the queuing times or queue lengths experienced by passengers. Thus it is difficult to judge if airlines are justified in complaining and difficult to assess the impact of a change in the allocation resulting from negotiation. Of course, the true effectiveness of a schedule can only be assessed retrospectively at the end of the week due to the stochastic nature of the problem; however, the aim of this study is to provide a tool which will give an indication of performance. This will assist the planner in making the initial allocation and in subsequent negotiation with the airlines. Figure 1 illustrative allocation of airlines to check-in desks When asked to help with this problem we felt that a discrete event visual interactive simulation model would be an appropriate analytic tool. Our aim was not to provide a tool which did the planner’s job for him, but one which facilitated and supported him in this task. Given a particular allocation schedule the simulation would utilise statistical distributions for arrival and service times of passengers to predict queuing patterns. The visual representation would facilitate communication with the airlines and other interested parties about the impact of particular allocations. A note about simulation Developing a simulation model nowadays is made easy by the availability of many high-level, visual interactive software systems. The developer no longer has to concern themselves with the details of event-by-event coding; the days of models which were difficult and time consuming to build and cumbersome to use are now past. It is now possible for the developer to work with a client interactively developing a model which can be run immediately. The modeller can concentrate on helping the client rather than on building the model. An initial model can be built very quickly, to be refined at a later stage if appropriate. A simulation is comprised of a logical model of the system being investigated, incorporating as many rules governing the behaviour of the system as are necessary to give an acceptable representation. The model also captures the stochastic nature of the system, representing variability by theoretical or empirical statistical distributions. Typically the user of a simulation model wants to test a number of possible scenarios and to gather information on the performance of the system in each instance. The simulation model is run a number of times for each scenario to generate a distribution of performance measures. The Model The simulation model was built using the simulation software system “SIMUL8” (Elder, 1995), to provide information for the planner on the expected performance of the system under different allocation schedules. It models the arrival of passengers at the airport, their decision about which queue to join and the time taken to check-in each passenger or group of passengers. The number of passengers, arrival times and check-in times, are generated randomly utilising available data for each flight type, i.e. business / leisure and domestic / international / long haul. The main simulation window, built together with the planner, depicts the physical layout of the check-in hall and the flow of passengers through it. A view of the model, annotated to assist the reader, is shown in Figure 2. When the model is running the display shows the length of each of the queues and the planner can see immediately if a particular allocation results in excessively long queues. This facility is important in helping the user to understand and informally validate the model, thereby developing ownership of the model and building confidence in the results it provides (Belton and Elder, 1994). The planner inputs to the model a particular allocation schedule of airlines and flights to check-in desks, such as the one illustrated in figure 1. The simulation takes this information from the spreadsheet and executes multiple runs to provide predictions of the following performance measures: ?? Number of flights for which any passenger queues more than 12 minutes ?? Number of flights for which more than 3% of passengers queue more than 12 minutes ?? Number of flights for which the average queuing time is greater than 4 minutes ?? Number of flights for which any queue length exceeds 10 passengers ?? Percentage of airline B’s passengers who queue for more than 6 minutes ?? Percentage of airline M’s passengers who queue for more than 6 minutes ?? Percentage of airline S’s passengers who queue for more than 9 minutes These measures were derived in discussion with the planner; the first two are ones which are specified by the parent company as corporate objectives; the last three relate to specific operators and on the basis of the planner’s experience are indicators of the likely level of complaints from the airlines; the remaining two measures are regarded as good indicators of general performance. At the time of writing the model is still under development and as the planner gains more experience with its use he may wish to add to, or change these. Such changes are easily incorporated. Figure 2 Main screen of the simulation model The model also provides the planner with the following information which helps him to see how a given plan might easily be improved: ?? The 10 flights which have the longest average queuing times ?? The 10 flights which have the longest maximum queuing time The planner can change the allocation schedule and re-run the simulation whenever he thinks he has a new schedule, which is worth evaluating. The simulation model was initially intended as a device to be used by the planner to find a good schedule. It was only later, when use of the model caused the investigation team to realise that the concept of a “good schedule” was a complex multicriteria issue, that the integration with MCDA became important. Integration of Simulation and MCDA From the discussion in the previous section of this paper, it is clear that the problem is not one for which there is an “optimal” solution. The planners are interested in multiple measures of performance, which cannot easily be reduced to a single dimension. Having been provided with more information to use as a basis for deciding how to allocate check-in desks, the planners are now faced with the multicriteria problem of identifying a preferred allocation. We decided to extend the system to draw on existing software for MCDA by integrating the simulation system (SIMUL8) with the multi-criteria decision support system, V•I•S•A. V•I•S•A is a system which supports the use of a multiattribute value function for multicriteria decision support. As with SIMUL8, the focus is on providing visual interactive facilities for problem representation and analysis. V•I•S•A utilises simple, easy to understand, visual displays to reflect back information and act as a catalyst for learning about the problem and about ones own and others' values. The visual interactive interface provides a powerful vehicle for exploring the implications of uncertainty about values and priorities. The alternatives to be considered here are the different allocation schedules suggested by the planners and the criteria for the evaluation of these schedules are initially taken to be the factors outlined above. Information about the performance of each allocation schedule is passed automatically to the multi-criteria model as the scenario is run through the simulation. The planners spend about one day each week drawing up the schedule for the following week; the following section illustrates the analysis of six schedules which they felt were worth considering further for that particular week. The multicriteria analysis Figure 3 illustrates the initial V•I•S•A model of the problem used to evaluate six proposed schedules. The seven measures outlined earlier are grouped into the three families indicated to give the value tree shown. The Alternatives Window shows the actual performance of each of the six test schedules against these measures; this data is imported directly from the simulation model. These performances are converted, within the V?I?S?A model, into value scores on a 0 to 100 scale (where 0 and 100 Figure 3 Illustration of the V•I•S•A model are defined as worst and best possible performance against each factor). Each of the criteria in the value tree are weighted to reflect acceptable trade-offs to the decision maker the cumulative weights reflecting the importance of the three toplevel criteria are shown in the window on the bottom left. The window on the bottom right shows the overall score for each of the six schedules and the window above that shows how that score is made up from performance on the three toplevel criteria. As can be seen from Figure 3, in this illustration the schedule named Friday Meeting is rated most highly overall, with Friday meeting B close behind. The profile graph shows that these two schedules perform substantially better than the others against “Corporate Objectives”, they are also the top two performers against “Mid-Range Measures”. Poorer performance against “Key Operator Queues”, particularly in the case of Friday Meeting B, is mitigated by the lower weight given to that criterion. The planners can use the software to interactively investigate the effect of changes to the criteria weights on the evaluation of the schedules. In this example, the weight on “Corporate Objectives” has to be significantly reduced to change the overall evaluation. When the planner creates a new schedule he wishes to consider, the information is picked up from the spreadsheet by the simulation package, which executes multiple runs and automatically creates a new alternative, together with performance measures, in V?I?S?A. However, the multicriteria analysis is not restricted to the measures generated by the simulation model; other factors may be introduced into the value tree and performance measures entered directly into V?I?S?A. For example, the planner may wish to take into account his subjective expectation of the likelihood that a schedule will be accepted by the airlines without complaint. Reflections This study is still ongoing and it is likely that both the simulation model and the multicriteria evaluation will be developed further in consultation with the client, a process that is made easy by the nature of the two softwares used. However, even at this stage of the process we feel that a number of important lessons have been learned. The linking of simulation and multicriteria analysis provides a decision support tool which spans the five categories of learning we have described elsewhere (Belton and Elder, 1994) as illustrated in Figure 4. Simulation provides the link between decision space and solution space (discovery), whilst multicriteria analysis provides the link between solution space and value space (clarification and explication). The overall analysis promotes understanding and creativity by allowing the user rapidly to try out new ways of doing things and immediately to see the consequences, which may lead the decision maker to change their view. This case study began with the simulation model and the multicriteria analysis was incorporated at a later stage. It may have been the case that a consideration of the multicriteria aspect of the problem, in particular the elicitation of relevant criteria, could have usefully informed the building of the simulation model. We anticipate that working with the V•I•S•A model will prompt the planners to think of other factors they would like to include in the evaluation, prompting an update of the simulation model in order to collect the appropriate performance measures. Figure 4: A Model of Learning We find it rather surprising that the integration of simulation and multicriteriaanalysis has not occurred more often. A simulation model more often than notprovides the user with multiple performance measures which are typically handledintuitively by the decision makers: MCDA is a natural extension of the analysiswhich can draw together all the elements and enable the decision makers to fullyappreciate the significance of the analysis. One barrier to integration may besoftware related although the link is easy to conceptualise it has not been easy toimplement using two separate software tools. However, this barrier has not beeninsurmountable in the past and it would not be difficult for simulation softwaresuppliers to integrate a tool, or tools, for MCDA. It would, perhaps, be instructivefor the MCDA community to reflect on why this has not happened but we willleave this as an issue for debate. REFERENCESBELTON, V., ACKERMANN, F. and SHEPHERD, I. (1997) "Integrated supportfrom problem structuring through to alternative evaluation using COPE andV•I•S•A” Journal of Multiple Criteria AnalysisBELTON, V. and ELDER, M.D. (1994). "Decision Support Systems Learningfrom Visual Interactive Modelling", Decision Support Systems (12), 355-364BELTON, V. and ELDER, M.D. (1996) “Exploring a Multicriteria Approach toProduction Scheduling”, Journal of the Operational Research Society (47), 162-174BELTON, V. and PICTET, J. (1997) Opinion Makers Section, Newsletter of theEuropean Wirkin Group on “Multicriteria Aid for Decisions”, Series 2, Number 11BELTON, V. and VICKERS, S.P. (1993) "Demystifying DEA: A Visual InteractiveApproach Based on Multicriteria Analysis", Journal of the Operational ResearchSociety (44), 883-896GRAVEL, M., MARTEL,J.M., NADEAU, R., PRICE, W. and TREMBLAY, R.(1991) "A Multicriterion View of Optimal Resource Allocation in Job-ShopProduction" presented to IFORS SPC1 on Decision Support Systems, Bruges.KORHONEN, P. (1997), Opinion Makers Section, Newsletter of the EuropeanWirkin Group on “Multicriteria Aid for Decisions”, Series 2, Number 10MACHARIS, C. (1997) “Hybrid Mmodelling: System Dynamics Combined withMulticriteria Analysis”, Vrije Universitet Brussels, Centre for Statistics andOperational Research, Working Paper.SPENGLER, T. and PENKUHN, T. (1996) “KOSIMEUS A Combination of aFlowsheeting Program and a Multicriteria Decision Support System” accepted forpublication in the Proceedings of SAMBA II Workshop (State of the Art ComputerPrograms for Material Balancing), Wuppertal Institut